As developers develop applications that increasingly utilize console resources, the optimization of the use of those resources has rapidly become a critical component of the operability of the application. One of the resources used in a console is system memory. A typical console may have several types of memory available for use by the application, some of which include the L1 and L2 cache, and main system memory. A developer has several means in which to determine the effectiveness of the use of the console's memory. L1 and L2 cache are used by the CPU as a means to reduce the average time to access memory. L1 and L2 cache are small units of memory that stores copies of data of recently accessed or frequently used by the CPU. The time to access memory is termed latency, and as the number of memory calls made to L1 or L2 cache made by an application, in lieu of memory calls to main memory, the average latency decreases, thus essentially increasing the speed at which an application may cause the console to process data.
One of the goals of a developer is to optimize the use of L1 and L2 cache. When an application accesses a memory location, the CPU first checks to see if the memory location is in the cache. L1 cache is typically faster and smaller than L2 cache. The CPU will typically check L1 cache first, then L2 cache, then if the system is not configured with another interim type of memory, will then access main memory. If the CPU checks the L1 or L2 cache and find the memory address in the cache, called a cache hit, the CPU accesses the data residing in the cache and continues processing. If the CPU checks L1 cache and does not find the memory address, called a cache miss, the CPU will then check L2 cache. If the memory address is not found in the L2 cache, again termed a cache miss, the CPU will then access the memory address in main memory, write the data into the cache, and then access the data written into the cache. The hit rate is the percentage of times which the CPU finds memory addresses in the cache as opposed to the times in which the CPU does not find the memory address in the cache. The hit rate assists developers in determining the efficiency of the application in using cache.
As still further way to determine the effectiveness of the use of memory is to determine the latency of memory calls. As discussed above, the time it takes for a CPU to read data from a memory address is latency. As the average relative latency increases, the speed at which an application processes data decreases, thus slowing the application. Latency is also undesirable because while memory is being retrieved, the CPU is typically idle. Developers measure the latency of memory accesses to determine problem areas of their application, then attempt to amend the source code of the application to better utilize the memory. There may be several reasons why an application has suboptimal latency results. For example, an application that accesses a significant number of different memory addresses on a regular basis may cause the cache memory addresses to constantly change so that the next time the address is accessed, the address will no longer be in the cache. Another reason may be that the memory address is one of multiple memory addresses that need to be accessed by a particular function. If the memory addresses are not co-located within the same memory blocks of the main memory, the CPU may need to run through the memory addresses to find the next memory address to be located. Another reason may be that a high number of cache misses are occurring. Each cache miss has a latent amount of time used when a cache miss occurs. The reduction of cache misses also reduces the CPU idle time, as memory accesses will typically be located in the fast CPU caches of the L1 and L2 as opposed to main memory.
Tools available to the developer may pinpoint what the inefficiencies are in a particular source code for an application, e.g. latency, hit rates, wasted bandwidth for memory caches, and bad cache utilization. Although important measures of the efficiency of memory usage, knowing what problems are occurring for an application may not provide the necessary degree of granularity to fine tune an application. In current consoles, because of the limitations of the particular system and expected application speed desired by the consumer, it is increasingly important to fully utilize memory resources.